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ABSTRACT 

Time delay and small capacity of 
communication are the primary constraint in 
super long distance telerobotic systems such as 
astronautical robotic tasks. Intelligent 
telerobotics is thought to break this constraint. 
We aim to realize this super long distance 
telerobotic system with object handling 
knowledge base and intelligent monitoring. We 
will discuss about physical and technical factor 
for this purpose. 

INTRODUCTION 

Telerobots such as space telerobotic systems 
use both autonomous and direct human control 
(manual control) in execution of their tasks. 
Supervisory control is a well known concept 
applied to these hybrid systems. From the 
viewpoint of flexibility and human friendliness 
in telerobotic task execution, we have proposed 
a more cooperative way to effectively utilize 
both autonomous functions of the robot and 
direct maneuvering by the human operator and 
developed MEISTER system[l]. 

In the telerobotic task execution visual 
information, such as TV monitors, is most 
important for cooperation between robots on a 
remote site and a human operator on a local 
site. However, in order that TV monitor 
supports a human operator effectively, the 
monitor should display scenes relevant to task 
situation. In the conventional systems, a human 
operator must control camera direction and 
viewing angle (zooming) manually along with 
robot task control. It increases burden of the 


operator severely. Therefore we have proposed 
intelligent control system of monitoring camera 
for telerobotic task execution[2]. 

First we will describe the principle of the 
intelligent monitoring system briefly, second 
discuss special issues of telerobotics executed 
over super long distance, and then show 
possible extension of the current intelligent 
monitoring functions for the issues. 

INTELLIGENT MONITORING 

The MEISTER system has a collection of 
task oriented object models as the knowledge 
base(3]. Each model contains environmental 
data which work as a world model. Handling 
knowledge, both generic and specific, is 
described by methods attached to object 
classes. 

The human operator achieves cooperation by 
watching the robot motion through TV 
monitors. Whether the human operator can help 
robot effectively or not depends on whether or 
not the TV monitors display appropriate scenes 
of task executions. In the original MEISTER 
system, the human operator controls viewing- 
point (camera direction) and viewing-angle 
(zooming) of the camera manually. We found 
that this controlling operation is the busiest part 
of the operator's task. Considering the 
problem, we have introduced intelligent 
monitoring in telerobotic task execution[2]. We 
call 'intelligent' to mean that a robot 
autonomously reports to the operator selected 
information relevant to a given task. 

The basic concept for the intelligent 
monitoring is based on the observation that we 
control our view according to what to see, 
when to sec, how to sec. These strategies seem 
to have deep relation with structure of 
manipulation tasks. 
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In the knowledge base of MEISTER, a pick 
operation is expanded into lower level motion 
commands such as 'approach,' 'm-t-g(move- 
to-grip),' 'grip,' and 'lift-up' motions depicted 
in Fig. 1. Differences in the meaning of these 
motions should correspond to different control 
strategies for monitoring action. This implies 


pick 



(a) Expansion of "pick" operation. 


place 



approach put-it-on ungrip depart 

(b) Expansion of "place" operation. 
Fig. 1. Expansion of pick-place operation. 


the operator should stop the robot or 
control the robot to avoid it. So in 
approach motion the monitoring camera 
should catch the whole area of the 
motion or follow the robot hand with 
viewing angle as wide as possible. 

m-t-g(move-to-grip) 

In this motion the robot hand moves to 
the grasping position of an object. It is 
a kind of guarded motion. Since the 
robot hand is already close to the 
object, the operator wants to look at the 
robot fingers and the object closely so 
that he or she can check their relative 
position. Therefore the camera should 
zoom in on the hand and the object. 

grip 

In this motion the robot hand does not 
change its position but its fingers close 
to hold the object. The aim of camera 
control is almost similar as the m-t-g 
motion. Further closing up helps the 
operator to confirm that the fingers 
hold the object successfully. 

lift-up 

The hand goes up into the free space to 
prepare next approach to another 
target. The camera should zoom out 
smoothly expecting this motion. 

put-it-on 

The hand sets the object in hand to the 
destination place. The camera should 
zoom in to cover the object in hand and 
the other one to which the former one 
will be assembled. 


understanding the contents of the task and how 
the task is proceeding. We apply this concept to 
the control of viewing direction and the viewing 
angle of the monitoring cameras so that the 
human operator can receive appropriate 
information to cooperate with the robot. The 
details of the strategies of camerawork for each 
motion are as follows. 

approach 

In this motion the robot hand 
approaches an object. This is a kind of 
global motion. The hand moves 
straight to the destination with a certain 
speed. If an obstacle exists on the path, 


ungrip 

The hand releases the object in hand. 
The camera is centered to fingers so 
that the operator can confirm that the 
object is successfully releases from the 
hand. 

depart 

This motion is similar to lift-up 
motion. The camera should zoom out 
smoothly covering current hand 
position and the point in the free space 
the hand goes to. 

PROBLEMS TO BE ATTACKED 
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Time delay and small capacity of 
communication are the hardest constraint in 
super long distance telerobotic systems. Direct 
power or position feed back loop between local 
and remote sites is not realistic because the time 
delay will be a few ten seconds or a minute. We 
cannot construct an efficient servo loop 
between local and remote sites. The problems 
are summarized into two points. 


Commanding Level 

An operator needs to command to robots 
with some high level robot language. If the 
commands which are sent from a local site to a 
remote site are much abstract, amount of 
communication will be decreased. 

On the other hand too high level commands are 
not sufficient to let an operator and robots 
execute tasks cooperatively and such a 
telerobotic system is not effective. Therefore it 
is also an important theme to determine 
command level corresponding to the degree of 
time delay of communication. 


Information Selection 

The operator should achieve not only 
commanding the robot but also watching the 
task environment with monitoring camera and 
various sensors in order to cooperate with the 
robot to execute tasks. So bi-directional 
communication is necessary between a local site 
of an operator and a remote site of robots. 

However not only time delay but also 
capacity of comunication are under constraint in 
super long distance telerobotic systems. It is 
not expected that all information can flow 
incontinently from the remote site to the local 
site. The remote system itself needs to select the 
information important to cooperate for the 
operator and send it to local site. 


MONITORING FUNCTIONS 

Basic strategies of intelligent monitoring 
described in the previous chapter will not be 
sufficient for the communication channel 
constraints problem. We propose following 
extensions for it. 


snapshot function 

Selecting and sending only important 
scenes when all the images can not be 
sent. Selection of viewing angles and 
viewing ranges in a sequence of task 
execution should be also included. 

simulation function 

Showing graphically simulated task 
procedure to give expected images of 
task status between the snapshots. 
These expected images help the 
operator to prepare response when the 
next new scene is displayed. 

confirmation function 

Confirming task status on each step 
using comparison of expected and real 
image on remote site. Though this is 
not directly monitoring, it can be 
seemed as an extension of monitoring 
of task procedure by robot itself. 

recording of whole video 

Storing whole scenes on remote site 
and send it to local site without any 
omission after the execution for 
analyzing errors later. This is a kind of 
telemetry. 

EXPERIMENT PLAN 
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Fig. 2 . Over pacific telerobotic operation. 
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Research topics described in this paper are 
for the collaborative research of ETL and JPL. 
In the collaborative research we plan to operate 
mutual telerobot testbeds (Fig. 2). 
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Fig. 3. Testbed plan at ETL. 


device such as joy-stick or master-manipulator 
will be also included for emergent intervention. 

For connection line between local and 
remote sites, we plan to try several ways such 
as inter-network, ordinary telephone-line with 
ISDN and/or conventional modem connection. 
These are to study about influence of quality of 
communication to telerobotic task execution. 

CONCLUDING REMARKS 

We discussed problems of super-long distance 
telerobotics, and plan to extend and apply the 
intelligent monitoring system. KHI (Kawasaki 
Heavy Industry Co.) collaborates with us to 
construct the testbed for this experiment. Detail 
of the construction of the testbed is presented in 
another paper. 

REFERENCES 

[l] Hirai, S., Sato, T. and Matsui, T., 

1992. Intelligent and Cooperative Control of 
Telerobot Tasks, IEEE Control Systems, 12(1): 
51-56. 


[2] Wakita, Y., Hirai, S. and Kino, T., 
1992. Automatic Camera-work Control for 
Intelligent Monitoring of Telerobotic Tasks. In 
Proceedings of the IEEE Intelligent Robots and 
Systems (IROS'92), 1 130-1 135.Raleigh, 
North Carolina, U.S.A. 

[3] Hirai, S. and Sato, T., 1988. Object 
Model for Telerobot. In Proceedings of the 
IEEE Intelligent Robots and Systems 
(IROS'88), 701-706. Tokyo, Japan. 


The outline structure of the experimental 
telerobot testbed under construction at ETL is 
illustrated in Fig. 3. 

Remote site subsystem includes a 
manipulator to execute task and monitoring 
camera(s) to monitor it. An image processing 
board is used to grab video images for 
monitoring, compress it to send to the local 
site. 

Local site subsystem includes an interface 
for object handling knowledge-base and 
graphics on UNIX workstations. Direct control 
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